Repository navigation
Async jobs improvements - #14270
Async jobs improvements#14270sureshanaparti wants to merge 56 commits into
Conversation
boring-cyborg
Bot
added
component:agent
component:api
component:compute
component:integration-test
component:integration
component:kvm
component:management-server
component:orchestration
component:UI
component:vmware
component:XenServer
Python
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Cancellation races can terminate refused jobs or finalize jobs while backend work continues, and several API/UI fields are ineffective or omitted.
Review effort: Balanced
Findings: 3
Open (6)
What changed in this PR
Adds async-job filtering, cancellation, event linkage, usage-job visibility, and an Activity-focused UI across CloudStack.
Changes:
- Adds cross-layer async-job cancellation for KVM, VMware, and XenServer.
- Extends job/event/usage APIs with filtering, status, duration, and linkage data.
- Adds Activity, resource job tabs, usage-job views, and automated tests.
| File | Description |
|---|---|
vmware-base/.../VmwareTaskRegistryTest.java |
Tests VMware task tracking and cancellation. |
vmware-base/.../VmwareTaskRegistry.java |
Tracks cancellable VMware tasks. |
vmware-base/.../VmwareClient.java |
Registers, cancels, and polls VMware tasks. |
utils/.../SerialVersionUID.java |
Adds cancellation exception UID. |
utils/.../CSExceptionErrorCode.java |
Registers cancellation error code. |
ui/.../UsageJobsTab.vue |
Adds usage-job table. |
ui/.../AsyncJobsTab.vue |
Adds filtering, pagination, and cancellation UI. |
ui/.../InstanceTab.vue |
Adds instance Jobs tab. |
ui/.../JobsView.vue |
Adds global Jobs view. |
ui/.../storage.js |
Adds volume Jobs tab. |
ui/.../managementServers.js |
Adds management and usage job tabs. |
ui/.../infra.js |
Moves alerts out of Infrastructure. |
ui/.../event.js |
Removes standalone Events section. |
ui/.../activity.js |
Adds Activity section. |
ui/.../router.js |
Registers Activity routing. |
ui/public/locales/en.json |
Adds job-related labels. |
test/.../test_async_jobs.py |
Adds async-job smoke coverage. |
server/.../UsageServiceImpl.java |
Implements usage-job listing. |
server/.../ManagementServerImpl.java |
Registers usage-job API. |
server/.../EventJoinDaoImpl.java |
Exposes event job IDs. |
server/.../ActionEventUtils.java |
Links events to current jobs. |
server/.../ConfigurationManagerImpl.java |
Validates cancellation interval. |
server/.../EventJoinVO.java |
Maps event job columns. |
server/.../AsyncJobJoinVO.java |
Maps completion server and related job. |
server/.../QueryManagerImpl.java |
Adds job/event filtering. |
server/.../AsyncJobJoinDaoImpl.java |
Reports executing/completing server. |
server/.../ApiResponseHelper.java |
Creates cancellation responses. |
server/.../ApiAsyncJobDispatcher.java |
Preserves terminal cancellation status. |
plugins/.../XenServerTaskRegistryTest.java |
Tests XenServer cancellation registry. |
plugins/.../CitrixResourceBaseTest.java |
Tests unknown XenServer requests. |
plugins/.../XenServerTaskRegistry.java |
Tracks XenAPI tasks. |
plugins/.../CitrixResourceBase.java |
Integrates XenServer cancellation. |
plugins/.../VmwareResourceTest.java |
Tests unknown VMware requests. |
plugins/.../VmwareResource.java |
Integrates VMware cancellation. |
plugins/.../LibvirtCancelCommandWrapperTest.java |
Tests KVM cancel wrapper. |
plugins/.../KvmCancellableRequestsTest.java |
Tests KVM request tracking. |
plugins/.../LibvirtCancelCommandWrapper.java |
Handles KVM cancel commands. |
plugins/.../LibvirtComputingResource.java |
Integrates KVM cancellation hooks. |
plugins/.../KvmCancellableRequests.java |
Tracks KVM cancellation hooks. |
framework/jobs/.../AsyncJobMonitor.java |
Tracks active job IDs. |
framework/jobs/.../AsyncJobManagerImpl.java |
Implements job cancellation lifecycle. |
framework/jobs/.../AsyncJobDaoImpl.java |
Queries related/cancelled jobs. |
framework/jobs/.../AsyncJobDao.java |
Extends job DAO contract. |
framework/jobs/.../AsyncJobManager.java |
Extends cancellation lifecycle API. |
framework/jobs/.../AsyncJobExecutionContext.java |
Handles missing join records. |
engine/schema/.../cloud.event_view.sql |
Adds event/job linkage. |
engine/schema/.../cloud.async_job_view.sql |
Adds related and completion fields. |
engine/schema/.../schema-42300to2400.sql |
Migrates event job linkage. |
engine/schema/.../EventVO.java |
Persists event job IDs. |
engine/orchestration/.../DirectAgentAttacheTest.java |
Tests direct-agent cancellation. |
engine/orchestration/.../ConnectedAgentAttacheTest.java |
Tests connected-agent cancellation. |
engine/orchestration/.../DirectAgentAttache.java |
Tracks and cancels direct tasks. |
engine/orchestration/.../ConnectedAgentAttache.java |
Sends agent cancel commands. |
engine/orchestration/.../AgentManagerImpl.java |
Coordinates job/agent cancellation. |
engine/orchestration/.../AgentAttache.java |
Adds cancellation-aware request handling. |
engine/components-api/.../AgentManager.java |
Exposes cancellation services. |
core/.../CancelCommandTest.java |
Tests cancellation probe mode. |
core/.../ServerResource.java |
Adds resource cancellation hooks. |
core/.../RequestExecutionContext.java |
Carries request sequence context. |
core/.../Request.java |
Tracks request cancellation. |
core/.../CancelCommand.java |
Adds probe-only cancellation mode. |
api/.../UsageService.java |
Exposes usage-job listing. |
api/.../JobInfo.java |
Adds explicit status values. |
api/.../JobCancellationHandler.java |
Defines cancellation handler contract. |
api/.../AsyncJobService.java |
Defines cancellation service/configuration. |
api/.../ResponseGenerator.java |
Adds cancellation response contract. |
api/.../UsageJobResponse.java |
Defines usage-job response. |
api/.../MigrateVolumeCmd.java |
Marks migration cancellable. |
api/.../DetachVolumeCmd.java |
Marks detach cancellable. |
api/.../DestroyVolumeCmd.java |
Marks volume destruction cancellable. |
api/.../CreateVolumeCmd.java |
Marks volume creation cancellable. |
api/.../AttachVolumeCmd.java |
Marks attach cancellable. |
api/.../RevertToVMSnapshotCmd.java |
Marks revert cancellable. |
api/.../DeleteVMSnapshotCmd.java |
Marks deletion cancellable. |
api/.../CreateVMSnapshotCmd.java |
Marks creation cancellable. |
api/.../StopVMCmd.java |
Marks stop cancellable. |
api/.../StartVMCmd.java |
Marks start cancellable. |
api/.../RebootVMCmd.java |
Marks reboot cancellable. |
api/.../DestroyVMCmd.java |
Marks VM destruction cancellable. |
api/.../BaseDeployVMCmd.java |
Declares deploy cancellation intent. |
api/.../RevertSnapshotCmd.java |
Marks snapshot revert cancellable. |
api/.../DeleteSnapshotCmd.java |
Marks snapshot deletion cancellable. |
api/.../CreateSnapshotCmd.java |
Marks snapshot creation cancellable. |
api/.../ListAsyncJobsCmd.java |
Adds status/date/duration filters. |
api/.../CancelAsyncJobCmd.java |
Adds cancellation API. |
api/.../ListEventsCmd.java |
Adds job-based event filtering. |
api/.../MigrateVolumeCmdByAdmin.java |
Marks admin migration cancellable. |
api/.../DetachVolumeCmdByAdmin.java |
Marks admin detach cancellable. |
api/.../DestroyVolumeCmdByAdmin.java |
Marks admin destruction cancellable. |
api/.../CreateVolumeCmdByAdmin.java |
Marks admin creation cancellable. |
api/.../AttachVolumeCmdByAdmin.java |
Marks admin attach cancellable. |
api/.../RevertToVMSnapshotCmdByAdmin.java |
Marks admin revert cancellable. |
api/.../StopVMCmdByAdmin.java |
Marks admin stop cancellable. |
api/.../StartVMCmdByAdmin.java |
Marks admin start cancellable. |
api/.../RebootVMCmdByAdmin.java |
Marks admin reboot cancellable. |
api/.../MigrateVMCmd.java |
Marks VM migration cancellable. |
api/.../MigrateVirtualMachineWithVolumeCmd.java |
Marks combined migration cancellable. |
api/.../DestroyVMCmdByAdmin.java |
Marks admin VM destruction cancellable. |
api/.../ListUsageJobsCmd.java |
Adds usage-job API. |
api/.../ApiConstants.java |
Adds usage-server constant. |
api/.../ApiCommandResourceType.java |
Adds Job resource type. |
api/.../APICommand.java |
Adds cancellable command metadata. |
api/.../VirtualMachine.java |
Adds Starting-to-Stopping transition. |
api/.../OperationCancelledException.java |
Adds cancellation exception. |
api/.../EventTypes.java |
Adds job-cancel event type. |
api/.../Answer.java |
Adds cancelled-answer state. |
agent/.../Agent.java |
Propagates request sequences. |
.github/workflows/ci.yml |
Runs new smoke tests. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #14270 +/- ##
============================================
+ Coverage 20.45% 20.54% +0.08%
- Complexity 21417 21645 +228
============================================
Files 6487 6501 +14
Lines 589329 591158 +1829
Branches 72246 72558 +312
============================================
+ Hits 120554 121432 +878
- Misses 455642 456463 +821
- Partials 13133 13263 +130
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@blueorangutan package |
|
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19353 |
|
@blueorangutan test keepEnv |
|
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
|
@blueorangutan test matrix |
|
@sureshanaparti a [SL] Trillian-Jenkins matrix job (ol8 mgmt + kvm-ol8, ubuntu22 mgmt + kvm-ubuntu22, ol8 mgmt + vmware-70u3, ol9 mgmt + xcpng82) has been kicked to run smoke tests |
|
[SF] Trillian Build Failed (tid-17049) |
|
[SF] Trillian test result (tid-17046)
|
|
[SF] Trillian test result (tid-17048)
|
|
@blueorangutan package |
|
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19649 |
|
@blueorangutan package |
|
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19671 |
|
@blueorangutan test keepEnv |
|
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
3c7c6a6 to
de9d19b
Compare
|
@blueorangutan package |
|
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19680 |
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
1 open issue
6 resolved since last review
Review effort: Lite
| public List<Long> getJobStatuses() { | ||
| if (jobStatuses == null) { | ||
| return null; | ||
| } | ||
|
|
||
| if (jobStatuses.isEmpty()) { | ||
| throw new InvalidParameterValueException("Empty job status"); | ||
| } | ||
|
|
||
| List<Long> statuses = new ArrayList<>(jobStatuses.size()); | ||
| for (String status : jobStatuses) { | ||
| statuses.add((long)parseJobStatus(status).value()); | ||
| } | ||
| return statuses; | ||
| } |
|
[SF] Trillian test result (tid-17215)
|
Description
This PR has the following async jobs related improvements.
More details here: https://cwiki.apache.org/confluence/spaces/CLOUDSTACK/pages/451979060/Async+Jobs+Improvements
Dor PR: apache/cloudstack-documentation#693
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?